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During the last few years, there has been a general 
in higher education on quality issues and the use of 
measures to assess institutional effectiveness. The quality 
in education suggests that previously defined performance 
are met through a review 



of outcomes assessment , 



This 

emphasis on outcomes measures represents a positive shift for 
community college occupational and technical programs, since, to a 
large extent, it is easier for occupational education to demonstrate 
effectiveness and accountability. An American College Testing survey 
of 675 colleges in 1990 found that the two most important measures of 
institutional effectiveness were employers' satisfaction with 
graduates and the percentage of students who received jobs. With 
respect to evaluating specific programs, performance standards should 
be based on identifiable and verified competencies which are 
determined in advance and explicitly stated. Four major steps for 
designing and implementing a performance standards system include 
defining student learning expectations in conjunction with customers, 
reviewing existing state or licensing agency standards, developing 
revising curricula to address the expected outcomes, and measuring 
outcomes with the goal of improving both the standards and 
curriculum. Finally, to maintain relevancy, standards must be 
constantly renewed to adjust to changes in the workplace and 
assessment methodologies of standards should be reviewed. (KP) 
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the private sector seems to concentrate less on the definition of quality and more on the process 
and the incremental achievement of quality activities with the idea that by striving for quality, the 
definition losses importance in the quest. 

The quality emphasis in education suggests that performance standards are met through a review 
of outcomes assessment New defmitions of quality focus on student outcomes, and in 
occupational education, students' abilities to effectively perform on the job. Yet, establishing 
outcomes assessment measures or performance standards can be costly. Tests or assessment 
instruments need to be developed and administered. Curriculum needs to be defmed in a 
competency mode. Curriculum needs to be reevaluated and revised consistently. Much more 
information on employer needs will need to be gathered, especially in a customer focus 
environment Achieving quality requires more than just tlie allocation of additional financial 
resources. Quality movements typically i^equire a whole new mindset-an entirely new way of 
doing business and an emphasis on educational outcomes rather than resources. In addition, 
quality should not be confused with establishing minimum performance standards. 

Goodwin (1989) suggested that there were five best predictors of occupational program quality. 
Intensity of instruction, defmed as "the extent to which faculty have tiie opportunity for in-depth, 
concentrated, and sustained instruction" was the most important predictor (p. 28). Programs that 
included a substantial portion of applied learning were valued highly by both students and 
employers. High quality programs are those that exhibit significant linkages with the private 
sector. Programs that were aggressive in their placement acti.vities, especially by faculty, were 
considered high in quality. And finally, programs tiiat provided accurate and honect public 
information were rated higher. 

Institutional Assessmf^nt and Student Outcomes 

The focus of institutional assessment has recently shifted from process oriented evaluation to 
outcome measures. Spurred by regional accrediting agencies, govemment and other funding 
bodies, and the public, institutional assessment has become more comprehensive and systematic. 
This shift is a positive direction for community college occupational and technical programs. 
To a large extent, it is easier for occupational education to demonstrate effectiveness and 
accountability. An American College Testing study on student outcomes surveyed 675 colleges 
(Cowart, 1990). The most important measurement of institutional effectiveness was employers' 
satisfaction with graduates. This measure was rated important or very unportant by 96% of the 
colleges responding to the survey. The second most favored measure of assessing academic 
progress was the percentage of students who received jobs, used by 70% of the colleges. Broad 
based outcome techniques have already been developed to assess the success of occupational 
training. As an example, the Washington State community and technical college system 
employees a state-wide computer match follow-up procedure on vocational students in 
conjunction with the state's Employment Security Department. Information is gathered on 
program leavers and completers to detennine employment status, placement in jobs or industries 
related to training, annual wages, fuU-or part-time employment, job tities, and continuing 
education status. 
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STUDENT OUTCOMES AND PERFORMANCE STANDARDS: 
ISSUES AND CHALLENGES FOR COMMUNITY AND TECHNICAL COLLEGES 



Walter H. Nolte, Ph.D. 



Introdtiction 

Throughout the last decade, student outcomes assessment, the accountability movement, and 
institutional effectiveness have been predominant trends in higher education. Competency based 
education linked to identifiable performance standards has also been an important trend in 
postsecondary vocational/technical education. Competency based instruction has been spurred by 
the Federal Vocational Act (Perkins H) and as a result of private sector concerns and demands for 
a trained and flexible workforce. Technology, the global economy, international competition, and 
the aging workforce are only a few of the factors changing the nature of work and training and 
education. The decline in the numbers of 18-24 year old entry level workers is having a profound 
effect on American business and industry. Businesses are beginning to hire employees whom they 
would not have considered five years ago (Owens & Linder, 1989). Significant numbers of 
workers are entering the workforce with less than adequate training (Nolte, 1991). The effect of 
these changes has resulted in national, state, and local movements embracing competency based 
education linked to performance standards that are verified by outcome assessment measures. 
Concern over the quality of education has also resulted because of scarce resources. This concern 
will increase as the nature of work continues to change, especially if the apparent mismatch 
between education and the workplace widens. 

Quality and customer service issues, like the Total Quality Management (TQM) movement are 
also in the forefront of new business techniques. These initiatives have and will continue to effect 
workplace education and training. This manuscript will provide an overview of the quality 
initiative and institunonal and outcomes assessment, differentiate between outcomes assessment 
and traditional program evaluation, highlight the need for performance standards m the context of 
vocational education, and suggest some basic criteria for the development oi performance 
standards. 

There is a new perspective or paradigm in the working world. "In the business sector, quality has 
replaced price and style as the chosen weapon in the market- share battle, in both America and 
foreign markets (Seymour, 1991, p. 1)." Defining quality in education, however, is problematic. 
What IS the definition of quality? Who determines what quality is? How will an institution or 
program know if it has achieved quality? What action can be taken to improve quality? Can 
quality be improved by initiating curriculum revision, especially in general education, or by 
establishing stringent admission standards, or by improving buildings or equipment? Is quality a 
value added issue? If so, how does this relate to performance standards? The TQM movement in 
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In a League for Innovation monograph, Seybert (1990) provided a guide for assessing the 
effectiveness of career preparation programs. The guide suggested data sources for a series of 
outcome oriented questions. Are college programs achieving educational outcomes as verified by 
competency testing, transfer rates, results of licensing and certifying exams, completion rates, or 
other competency based measures? Are student? fulfilling their educational goals as verified by 
employment, wages, and transfer rates? Are employers' needs met as verified by surveys of 
employers? Are programs contributing to the economic health of the community, as 
demonstrated by the number of employers attracted to a community or economic impact studies? 

At the institutional level, measures of effectiveness can be multiple. Many schools gather 
information and analyze course and program completion rates, student grade point averages, 
degrees awarded by p-ograms, and other follow-up data to determine critical success factors. 
Information provided by state or regional data sources should be examined and analyzed in the 
development of a performance standard system. Enrollment and demographic data, transfer 
information, program evaluation and accreditation reports, entry/exit tests, regional accreditation 
reports, student evaluations of courses and instructors, and licensure and certification results are 
all common measures of effectiveness. Except for licensure tests results, most of these do not 
directiy address performance standards. 

Program Evaluation 

Traditional evaluation of occupational program effectiveness often deals with input-output 
models. Inputs-funding, staff, buildings, and equipment-are analyzed based on the 
organizational structure, curriculum, and method of instruction and compared to output as 
measured by student learning, value added to the student and the economic resources of the 
community, and advisory committee and employer satisfaction. Increases in earning power, 
achievement, or pass rates on standardized exams are compiled and integrated with information 
on enrollment trends, cost per student, student/faculty ratios, or total cost of a program. 
Evaluations may include both quantitative and qualitative approaches. Interviews with students, 
faculty, and employers, class or laL observations, logs or diaries are all qualitative approaches. 
These evaluations may address tPie need to assess student learning and outcomes, but may not 
address how students are doing in relation to an approved standard or accountability measure. 
They are often an evaluation of the instructional process, not student performance. They often 
address the appropriateness of a program or discrepancies between program goals and outcomes 
or if a program produces a trained employee in a more effective or efficient manner. As indicated 
above, the institutional effectiveness movement has shifted from a process oriented evaluation to 
an outcomes orientation, focusing on results, accomplishments, and skills. Outcomes assessment 
requires planning-a need to set direction and purpose and establish the expected performance 
standards. Review of performance standards and analysis of the outcomes should focus on 
improving institutional effectiveness and curriculum adaptation. 

According to Peter Ewell (1983) program review and evaluation is a systematic, regular, and 
comprehensive process that involves making a judgment on student outcomes. Ewell argued that 
program review should focus on educational outcomes rather than more traditional evaluation 
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techniques that deal with inputs, resources, and efficiency. This move to outcomes and 
accountability is the result of a significant number of new postsecondary institutions and a rising 
number of students from all ages and background attending college during the last thirty years. 
Ewell asked. "What has the program produced" (1983, p. 4), He describes the program review 
process as similar to the production cycle. An unfinished product is educated, becomes a finished 
product witli value added. The notion of outcomes assessment is to measure the value-added 
product. In recent years, this measurement technique has been compared to criteria based 
performance standards tiiat relates to common industry standards on employee skills and abilities. 
The concept is akin to tiie private sector ideas of total quality control. Ewell suggested a variety 
of achievement indicators. These range from program completion rates, job market needs, or 
incomes of program completers or graduates. Program review concentrates on student questions. 
Do students know more when they have finished a program? Can tiiey do anytiiing better? What 
do students feel about their educational experience? This is an outcomes approach, however, it is 
an approach that does not relate outcomes to performance in the workplace. Ewell was quick to 
point out that program evaluation is judgmental, a human process trying to find "...the elusive 
dimension of quality (p. 1 1)." In anotiier work, Ewell suggested that, "Even the most reliable 
quantitative data measurement of outcomes is at best only an indicator of the underlying attribute 
to be assessed (Ewell, 1983a, p.2)." 

Perfoi mance Standards 

'^e movement to establish educational performance standards for occupational programs is 
designed to assure that what is being taught is what is needed in the workplace. Is a broad or 
narrow curriculum needed-a practical hands on program that very early provides real life 
experiences or a theoretical or abstract based program with little practical training? Similar to 
defining quality, however, clarifying performance standards is also problematic. Hoachlander 
(1991) defined performance standards as 

tiie level of performance considered acceptable for (1) advancing to tiie next level of 
education or training, (2) successfully entering a job related to training, (3) continuing to a 
course or program, or (4) establishing acceptable program or institutional performance- 
e.g., correctiy completing an overhaul of front disc brakes witiiin 90 minutes, 75 percent 
of students mastering correct brake overhaul within one semester of instruction, 70 
percent of students choosing to enter an automotive program completing the program, 70 
percent of students opting for vocational education completing a sequence of vocational 
and academic courses (p. 1). 

Hoachlander suggested that standards can be defined for students, courses, programs, institutions, 
districts, regions, or states and can be established to measure performance at a single point or 
over time. Standards should be clearly defined, easy and inexpensive to measure, consistent, and 
quantifiable, according to Hoachlander. In addition, the number of measures must be manageable 
and administered on a consistent and regular timetable to provide credibility. 
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There are several difficulties with establishing effective performance standards. Defining 
perfonmnce standards for die soft or adaptive skills is also difficult For example, how are 
changes in student attitudes road values measured? The question of insritiational autonomy or local 
conditions is complex when dealing with performance standards for a specific occupation or trade. 
Faculty also have issues and concerns with performance standards. They raise the academic 
freedom question, die measurement of soft skills, the amount of work necessary to develop 
performance standards, or they question will it really matter-is it only a fad? Are performance 
standards really an oblique rnethod of faculty evaluation? Faculty and other college personnel 
may fear that programs will be unable to graduate enough students to meet performance 
standards. 

There are also problems witii nationally normed objective assessment instruments. Many measure 
only recall and recognition skills, not higher level cognitive skills (analysis or synthesis). Also, 
testing can drive instruction-will perfomance standards drive instruction, excluding other forms 
of learning? Who evaluates tiie quality of the assessment instrument? Are performance standards 
based on die entire occupational responsibility, or die ones existing at the time the standard was 
written? Is tiiere a model curriculum associated with the standard? Does the curriculum leave 
room for local variabiUty? Does the standard leave room for local variability? 

There are four major steps for designing and implementing a performance stiindards system: 1) 
define student learning expectations in conjunction with customers, 2) review existing standards 
such as state approved measures or standards provided by licensing or certification agencies or 
associations, 3) develop or revise curriculum to address tiie expected outcomes, 4) measure 
outcomes with a goal of improving both the standards and the curriculum. 

In an effort to promote technical education and to assist business and industry in becoming more 
productive and competitive, the American Technical Education Association published a set of 
national minimum standards for technical associate degree programs in 1991. These standards 
were designed to prepare technical students to ootain a job, become productive workers with little 
additional tinning, and advance with the technical developments in an occupational field. The 
Association suggests that 49% of the curriculum should be technical core courses, 15% general 
education communications and social studies courses, 13% math and science support courses, 
10% related technical courses, and 13% controlled electives. 

Performance standards do not remain static, some occupations will need minor adjustments from 
year-to-year-others will need major rework if substantial changes occur in the v/orkplace, 
especially related to technology. If standards are not kept current, they will not l^e taken seriously 
ana ultimately discontinued or allowed to lapse. Assessment methodologies of standards should 
also be reviewed on a consistent timeline, as new methods are developed. Perfoimance standards 
must also be assessed in a way to permit accurate measurement of smdents' skills and abilities and 
must account for the different ways students leam and communicate. Thus, assessment of the 



standard might require written, oral, or visual measurement techniques. And finally, Elman and 
Lynton (1986) suggested that 

If performance based assessment is to become an important element in career-oriented 
education, it is necessary to incorporate real or simulated experiences as earlier and more 
integral components of the curriculum, and to structure them in such a way as to provide 
valid opportunities for assessment (p. 68). 

Conclusion 

Performance standards should be based on identifiable and verified competency. These 
competencies should be determined in advance and explicitly stated. The competencies should 
demonstrate accomplishments and then assessment techniques can include actual performance. 
Students can move through the competencies in a simulated work environment at their own pace, 
receiving meaningful feedback on their work. The private sector should be intimately involved in 
the development of the competencies and final performance standards. This involvement will 
assist the private sector in recruiting and retaining qualified employees. The involvement will also 
solidify a trusting relationship between business and education. 

As indicated, during the last few years, there has been a shift in emphasis by regional accrediting 
agencies to the use of outcome measures to assess institutional effectiveness. This '^hift provides 
community and technical colleges an opportunity to focus on accountability and to demonstrate 
program effectiveness to college stakeholders-the public, the federal government, state and local 
funding bodies, and students. The shift also offers occupational and technical education the 
opportunity to utilize industry validated performance standards to improve teaching and learrang 
and to systematically demonstrate a need for additional resources. The development of 
performance standards allows colleges the opportunity to accurately measure students' skills and 
abilities and to focus on what works, not the process to achieve what works. The development of 
occupational specific performance standards will not be easy. There are enormous costs 
associated with both the development and implementation of standards and the development and 
implementation of curricula to address the standards. 
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